Head-mounted display apparatus worn on user&#39;s head

ABSTRACT

A head-mounted display apparatus worn on the head of a user and a method for controlling the head-mounted display apparatus are disclosed. The head-mounted display apparatus includes an attitude detector that detects the attitude of the head-mounted display apparatus, a manipulation detector that detects a predetermined manipulation performed on the head-mounted display apparatus, a display, and a processor. The head-mounted display apparatus switches a display of first display information to second display information according to a change in the attitude detected by the attitude detector. However, if the manipulation detector detects the predetermined manipulation, the head-mounted display apparatus maintains the display of the first display information without switching to the second display information even if there is a change in the attitude detected by the attitude sensor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation application of pending U.S. patent applicationSer. No. 15/131,902, filed on Apr. 18, 2016, which claims priority toJapan Application No. 2015-089815 filed Apr. 24, 2015. The disclosuresof these documents, including the specifications, drawings and claims,are incorporated herein by reference in their entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a head-mounted display apparatus and amethod of controlling it.

2. Description of the Related Art

A display apparatus is worn on the head of a user so that the user canview an image (this type of display apparatus will be referred to belowas a head-mounted display apparatus). By using the head-mounted displayapparatus, the user can view an image without using a hand or changingthe direction of the field of view of the user. Therefore, thehead-mounted display apparatus is very useful when, for example, amaintenance person services a machine while viewing an image thatindicates a maintenance procedure.

International Publication No. WO 2014/128751 discloses a head-mounteddisplay apparatus that presents information to a user as desired by theuser according to the attitude of the head-mounted display apparatus.With the head-mounted display apparatus disclosed in InternationalPublication No. WO 2014/128751, the user can automatically obtaindesired information quickly with ease.

SUMMARY

With the technology disclosed in International Publication No. WO2014/128751, however, the head-mounted display apparatus switchesinformation to be presented to the user independently of the user'sintention. Therefore, information to be displayed on the head-mounteddisplay apparatus may be changed contrary to the user's intention.

One non-limiting and exemplary embodiment provides a head-mounteddisplay apparatus and the like that appropriately present information toa user as desired by the user.

In one general aspect, the techniques disclosed here feature ahead-mounted display apparatus worn on the head of a user; thehead-mounted display apparatus includes an attitude sensor that detectsan attitude of the head-mounted display apparatus, a manipulation sensorthat detects a predetermined manipulation performed on the head-mounteddisplay apparatus, a display that displays information, a processor, anda memory having a program stored therein, the program causing theprocessor to execute operations in which information to be displayed onthe display is switched according to a change in the attitude detectedby the attitude sensor, and when the manipulation sensor detects thepredetermined manipulation, even if there is a change in the attitudedetected by the attitude sensor, information being displayed on thedisplay is maintained without being switched.

The head-mounted display apparatus in the present disclosure canappropriately present information to a user as desired by the user.

It should be noted that these comprehensive or specific aspects may beimplemented as a system, a method, an integrated circuit, a computerprogram, a recording medium such as a computer-readable compactdisc-read-only memory (CD-ROM), or any selective combination thereof.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an outside shape of a head-mounted display apparatusin a first embodiment;

FIG. 2 is a block diagram illustrating the hardware structure of thehead-mounted display apparatus in the first embodiment;

FIG. 3 is a block diagram illustrating the functional structure of thehead-mounted display apparatus in the first embodiment;

FIG. 4 illustrates how an attitude detector in the first embodimentdetects an attitude;

FIG. 5 also illustrates how the attitude detector in the firstembodiment detects an attitude;

FIG. 6 illustrates a table, in the first embodiment, that indicatesattitude information and types of display information displayed by acontroller in correspondence to each other;

FIG. 7 illustrates, in the first embodiment, the field of view of theuser and icons displayed on a displayer;

FIG. 8 illustrates, in the first embodiment, the field of view of theuser and an image displayed on the displayer;

FIG. 9 illustrates a highlighted display form in the first embodiment;

FIG. 10 is a flowchart of processing performed by the controller in thefirst embodiment to switch or maintain display information;

FIG. 11 is a flowchart of processing performed by the controller in thefirst embodiment to terminate the maintaining of a display;

FIG. 12A illustrates a controller in the first embodiment whichmaintains a display and terminates the maintaining of the display;

FIG. 12B illustrates a controller in the first embodiment whichmaintains a display and terminates the maintaining of the display;

FIG. 12C illustrates a controller in the first embodiment whichmaintains a display and terminates the maintaining of the display;

FIG. 12D illustrates a controller in the first embodiment whichmaintains a display and terminates the maintaining of the display;

FIG. 12E illustrates a controller in the first embodiment whichmaintains a display and terminates the maintaining of the display;

FIG. 13 illustrates display information, in the first embodiment, thatincludes a large number of images;

FIG. 14 illustrates display information, in the first embodiment, thatincludes a small number of characters;

FIG. 15 illustrates display information, in the first embodiment, thatincludes a large number of characters;

FIG. 16 is an outside shape of a head-mounted display apparatus in asecond embodiment;

FIG. 17 is a block diagram illustrating the hardware structure of thehead-mounted display apparatus in the second embodiment;

FIG. 18 is a block diagram illustrating the functional structure of thehead-mounted display apparatus in the second embodiment;

FIG. 19 is a flowchart of processing performed by the controller in thesecond embodiment to switch a display;

FIG. 20A illustrates display information in the second embodiment;

FIG. 20B illustrates display information in the second embodiment; and

FIG. 20C illustrates display information in the second embodiment.

DETAILED DESCRIPTION

Underlying Knowledge Forming Basis of the Present Disclosure

The inventor found that a problem with the head-mounted displayapparatus described in Description of the Related Art arises asdescribed below.

When a person looks up at the sky, the person often worries about theweather. If a head-mounted display apparatus displays a mark thatindicates the today's weather when a user looks up at the sky, it may bepossible to achieve an intuitive manipulation by the user. Aconventional head-mounted display apparatus based on this idea switchesinformation to be presented to a user according to a change in theattitude of the head-mounted display apparatus (see InternationalPublication No. WO 2014/128751).

According to the technology disclosed in International Publication No.WO 2014/128751, when the attitude of the head-mounted display apparatus(direction in which the head-mounted display apparatus is inclined) ischanged, information to be presented to the user changes. Therefore, ifthe information to be presented to the user is such that the user cangrasp the information in a relatively short time (such as, for example,a single sunny mark indicating the weather of a day), the user can causethe head-mounted display apparatus to display the information by anintuitive motion to switch display information according to the attitudeof the head-mounted display apparatus.

However, information to be presented to the user may be such that ittakes a relatively long time for the user to grasp the information (suchas, for example, a time trend of probability of rain). When this type ofinformation is displayed in the method described above by switching thedisplay in a similar manner according to a change in the attitude of thehead-mounted display apparatus, if the attitude of the head-mounteddisplay apparatus changes before the user completely grasps theinformation, the display is switched, in which case the user may beunable to grasp the information. That is, if the user wants to continueto view the information, the user needs to maintain the head-mounteddisplay apparatus in a fixed attitude, lowering the convenience of thehead-mounted display apparatus.

If information to be presented to the user changes contrary to theuser's intention, the user needs to make another motion to restore thehead-mounted display apparatus to the previous attitude. In addition,the head-mounted display apparatus needs to perform processing to switchinformation to be presented to the previous information according to theabove user's motion.

In view of the above situation, the present disclosure provides ahead-mounted display apparatus and the like that appropriately presentinformation to a user as desired by the user without the head-mounteddisplay apparatus having to be maintained in a fixed attitude.

To solve the above problem, the head-mounted display apparatus in oneaspect of the present disclosure is a head-mounted display apparatusworn on the head of a user; the head-mounted display apparatus includesan attitude sensor that detects an attitude of the head-mounted displayapparatus, a manipulation sensor that detects a predeterminedmanipulation performed on the head-mounted display apparatus, a displaythat displays information, a processor, and a memory having a programstored therein, the program causing the processor to execute operationsin which information to be displayed on the display is switchedaccording to a change in the attitude detected by the attitude sensor,and when the manipulation sensor detects the predetermined manipulation,even if there is a change in the attitude detected by the attitudesensor, information being displayed on the display is maintained withoutbeing switched.

Accordingly, when the user fixes (maintains) displayed information byperforming a predetermined manipulation on the basis of the user's owndecision, the head-mounted display apparatus can display necessaryinformation regardless of the attitude of the head-mounted displayapparatus. As described above, the head-mounted display apparatus canappropriately present information to a user as desired by the user.

An ordinary head-mounted display apparatus displays information only fora user wearing the head-mounted display apparatus. Therefore, it is saidthat a user who is wearing a head-mounted display apparatus and isperforming a manipulation for the head-mounted display apparatus appearsto be strange to persons in the vicinity of the user. If information canbe displayed on the head-mounted display apparatus by the above motion,the user can display necessary information on the head-mounted displayapparatus through a natural motion without making persons around theuser feel uncomfortable.

As described above, the user not only can collect information through anintuitive natural motion without making persons around the user feeluncomfortable but also can grasp information that is such that it takesa relative long time for the user to grasp the information.

In the operations described above, for example, when the manipulationsensor detects the predetermined manipulation, if there is a change inthe attitude detected by the attitude sensor, information beingdisplayed on the display is maintained without being switched and thedisplay form of the information is changed.

Accordingly, only by changing the display form of the information thatis being displayed by the head-mounted display apparatus, the user canrecognize at a glance that the displayed information is fixed regardlessof a change in the attitude of the head-mounted display apparatus. Thiscan prevent the user from mistakenly recognizing a situation in whichthe attitude of the head-mounted display apparatus has been changed butthe display is not switched as a failure of the head-mounted displayapparatus.

In the operations described above, for example, when the information isdisplayed on the display, a figure is displayed as the background of theinformation together with the information, and the display form of theinformation is changed by changing the color of the figure.

In the operations described above, for example, when the information isdisplayed on the display, a frame that encloses the information isdisplayed together with the information, and the display form of theinformation is changed by changing the thickness or color of the frame.

Accordingly, by changing the color of the background or frame that isbeing displayed by the head-mounted display apparatus, the user canspecifically recognize that a display given by the head-mounted displayapparatus is fixed.

In the operations described above, for example, it is decided whetherthe amount of information to be displayed on the display is larger thana predetermined value. Only if the amount of information is decided tobe larger than the predetermined value, when the manipulation sensordetects the predetermined manipulation, the information being displayedon the display is maintained without being switched even if there is achange in the attitude detected by the attitude sensor.

In the operations described above, for example, it is decided whetherthe amount of information to be displayed on the display is smaller thanthe predetermined value. If the amount of information is decided to besmaller than the predetermined value, information to be displayed on thedisplay is switched according to a change in the attitude detected bythe attitude sensor even if the manipulation sensor detects thepredetermined manipulation.

Accordingly, according to the feature of the information beingdisplayed, the head-mounted display apparatus decides whether a displayof the information is maintained or switched according to the attitudeof the head-mounted display apparatus. Specifically, it is possible toinfer to a certain extent whether the information is such that the userwants to switch the display of the information according to the attitudeof the head-mounted display apparatus (that is, the information is suchthat the user can grasp the information in a relatively short time) orthe information is such that the user does not want to switch thedisplay of the information (that is, the information is such that ittakes a relatively long time for the user to grasp the information). Foreach piece of information, therefore, the head-mounted display apparatuscan set in advance which of the above two types the information is.

Thus, even if the user performs a predetermined manipulation by mistakein a state in which information that the user can grasp in a relativelyshort time (such as, for example, a single sunny mark indicating theweather of a day) is displayed, it is possible to switch information tobe displayed according to the attitude of the head-mounted displayapparatus.

In the operations described above, for example, if the manipulationsensor detects the predetermined manipulation and further detects asubsequent manipulation that is the same as the predeterminedmanipulation, information to be displayed on the display is switchedaccording to the attitude detected by the attitude sensor when themanipulation sensor has detected the subsequent manipulation.

Accordingly, when the head-mounted display apparatus terminates thestate in which a display is fixed, the head-mounted display apparatuscan display appropriate information according to the attitude of thehead-mounted display apparatus without the user having to perform somemanipulation. This reduces a load on the user when the user views nextinformation.

In the operations described above, for example, if the manipulationsensor detects the predetermined manipulation and further detects asubsequent manipulation that is the same as the predeterminedmanipulation, the displaying of information on the display is stopped.

Accordingly, when the head-mounted display apparatus terminates a statein which a display is fixed, the head-mounted display apparatus deletesthe display to prevent the user from displaying information that theuser does not need. This reduces annoyance that the user feels whenunnecessary information is displayed.

For example, the manipulation sensor has a manipulation button; themanipulation sensor detects, as the predetermined manipulation, thepressing of the button or a touch to it.

For example, the manipulation sensor has a touchpad; the manipulationsensor detects a tap on the touchpad as the predetermined manipulation.

For example, the manipulation sensor has a microphone and a voicerecognizing apparatus that acquires a speech of the user through voicerecognition of voice collected by the microphone; the manipulationsensor detects, as the predetermined manipulation, the acquisition of apredetermined speech by the voice recognizing apparatus.

Accordingly, when the user presses the manipulation button, taps thetouchpad, or makes a voice input, the head-mounted display apparatus canspecifically accept a manipulation from the user.

For example, the attitude sensor creates attitude information, whichindicates a direction to which the attitude has been changed from areference attitude, the direction being one of an upward direction, adownward direction, a leftward direction, and a rightward direction whenviewed from the user. In the operations described above: fourinformation items associated with the upward, downward, leftward, andrightward directions are stored in the memory as information to bedisplayed on the display; icons are displayed at portions, on thedisplay area of the display, in the upward, downward, leftward, andrightward directions, each icon indicating one of the four informationitems that is associated with the upward direction, downward direction,leftward direction, or rightward direction, whichever is applicable; andwhen the attitude sensor creates the attitude information, informationassociated with the direction indicated by the created attitudeinformation is displayed on the display.

Accordingly, since the head-mounted display apparatus displays an icon,the user can recognize both a direction in which the head-mounteddisplay apparatus is inclined and information assigned to the direction.This eliminates the need for the user to remember that what informationis assigned to what direction in which the head-mounted displayapparatus is inclined, reducing a burden on the user.

For example, the head-mounted display apparatus further has a gesturesensor that detects a swipe gesture manipulation that indicates onedirection, the swipe gesture manipulation being performed in amanipulation space. In the operations described above, information to bedisplayed by the display is switched according to the directionindicated by the swipe gesture manipulation.

Accordingly, the user can switch display information through a swipemanipulation instead of changing the attitude of the head-mounteddisplay apparatus. Therefore, for example, while facing the front, theuser can cause the head-mounted display apparatus to display necessaryinformation through a swipe manipulation and can grasp the informationand can grasp the information; the necessary information would otherwisebe displayed by changing the attitude of the head-mounted displayapparatus. At that time, the user does not need to change the attitudeof the head-mounted display apparatus. It is possible to increasechoices of user's manipulations to check information that is such thatit takes a relative long time for the user to grasp the information.

A head-mounted display apparatus in one aspect of the present disclosureis a head-mounted display apparatus worn on the head of a user; thehead-mounted display apparatus includes an attitude sensor that detectsan attitude of the head-mounted display apparatus, a gesture sensor thatdetects a swipe gesture manipulation that indicates one direction, theswipe gesture manipulation being performed in a manipulation space, adisplay that displays information, a processor, and a memory having aprogram stored therein, the program causing the processor to executeoperations in which information to be displayed on the display isswitched according to a change in the attitude detected by the attitudesensor, and information to be displayed on the display is switchedaccording to the direction indicated by the swipe gesture manipulationdetected by the gesture sensor.

Accordingly, the user can switch display information by performing aswipe manipulation instead of changing the attitude of the head-mounteddisplay apparatus. Therefore, for example, while facing the front, theuser can cause the head-mounted display apparatus to display necessaryinformation through a swipe manipulation and can grasp the information;the necessary information would otherwise be displayed by changing theattitude of the head-mounted display apparatus. As described above, thehead-mounted display apparatus can appropriately present information toa user as desired by the user.

As described above, the user not only can collect information through anintuitive natural motion without making persons around the user feeluncomfortable but also can grasp information that is such that it takesa relative long time for the user to grasp the information.

For example, the gesture sensor detects a motion of a hand of the userfrom above to down as a swipe gesture manipulation indicating thedownward direction, detects a motion of the hand of the user from downto above as a swipe gesture manipulation indicating the upwarddirection, detects a motion of the hand of the user from left to rightas a swipe gesture manipulation indicating the rightward direction, anddetects a motion of the hand of the user from right to left as a swipegesture manipulation indicating the leftward direction. In theoperations described above, information to be displayed on the displayis switched according to the direction indicated by the swipe gesturemanipulation detected by the gesture sensor.

Accordingly, the head-mounted display apparatus can accept a gesturemanipulation performed by the user as a manipulation that indicates oneof the four directions, which are the upward, downward, leftward, andrightward directions, and can display information corresponding to theaccepted direction. Since the number of directions and the number ofinformation items are reduced in this way, a manipulation by a roughgesture can be accepted without requiring the user to perform a strictmanipulation, so manipulability can be improved.

For example, the attitude sensor creates attitude information, whichindicates a direction to which the attitude has been changed from areference attitude, the direction being one of the upward, downward,leftward, and rightward directions when viewed from the user. In theoperations described above: four information items associated with theupward, downward, leftward, and rightward directions are stored in thememory as information to be displayed on the display; icons aredisplayed at portions, on the display area of the display, in theupward, downward, leftward, and rightward directions, each iconindicating one of the four information items that is associated with theupward direction, downward direction, leftward direction, or rightwarddirection, whichever is applicable; and when the gesture sensor detectsthe swipe gesture manipulation, information associated with thedirection indicating the start point of the swipe gesture manipulationis displayed on the display.

Accordingly, since the head-mounted display apparatus displays an icon,the user can recognize both the direction of a gesture manipulation andinformation assigned to the direction. This eliminates the need for theuser to remember that what information is assigned to what direction ofa gesture manipulation, reducing a burden on the user.

For example, the gesture sensor has a camera that captures images of ahand of the user with visible light; the gesture sensor detects theswipe gesture manipulation according to the motion of the hand of theuser, the images of the hand having been captured by the camera.

For example, the gesture sensor has an infrared sensor that detectsinfrared rays released from a hand of the user; the gesture sensordetects the swipe gesture manipulation according to the motion of thehand of the user, the motion having been acquired by the infraredsensor.

Accordingly, the head-mounted display apparatus can acquire a motion ofa hand of the user with a visible camera or an infrared camera and canspecifically accept a gesture manipulation according to the motion.

For example, the head-mounted display apparatus further has amanipulation sensor that detects a predetermined manipulation performedon the head-mounted display apparatus; when the manipulation sensordetects the predetermined manipulation performed by the user, even ifthere is a change in the attitude detected by the attitude sensor,information being displayed on the display is maintained without beingswitched.

Accordingly, when the user fixes (maintains) displayed information byperforming a predetermined manipulation on the basis of the user's owndecision, the head-mounted display apparatus can display necessaryinformation regardless of the attitude of the head-mounted displayapparatus.

As described above, the user not only can collect information through anintuitive natural motion without making persons around the user feeluncomfortable but also can grasp information that is such that it takesa relative long time for the user to grasp the information.

A head-mounted display apparatus control method in one aspect of thepresent disclosure is a method of controlling a head-mounted displayapparatus worn on the head of a user, the head-mounted display apparatushaving a display that displays information. The head-mounted displayapparatus control method includes: an attitude detection step ofdetecting an attitude of the head-mounted display apparatus by using anattitude sensor; a manipulation detection step of detecting apredetermined manipulation performed on the head-mounted displayapparatus by using a manipulation sensor; and a control step of causinga processor to switch information to be displayed on the displayaccording to a change in the attitude detected in the attitude detectionstep. In the control step, when the predetermined manipulation isdetected in the manipulation detection step, even if there is a changein the attitude detected in the attitude detection step, informationbeing displayed on the display is maintained without being switched.

Thus, the same effect as with the head-mounted display apparatusdescribed above can be obtained.

A head-mounted display apparatus control method in one aspect of thepresent disclosure is a method of controlling a head-mounted displayapparatus worn on the head of a user, the head-mounted display apparatushaving a display that displays information. The head-mounted displayapparatus control method includes: an attitude detection step ofdetecting an attitude of the head-mounted display apparatus by using anattitude sensor; a gesture detection step of detecting a swipe gesturemanipulation that indicates one direction by using a gesture sensor, theswipe gesture manipulation being performed in a manipulation space; anda control step of causing a processor to switch information to bedisplayed on the display according to a change in the attitude detectedin the attitude detection step. In the control step, information to bedisplayed on the display is switched according to the directionindicated by the swipe gesture manipulation detected by the gesturesensor.

Thus, the same effect as with the head-mounted display apparatusdescribed above can be obtained.

A head-mounted display apparatus according to an embodiment of thepresent disclosure will be described below with reference to thedrawings.

It should be noted that these comprehensive or specific aspects may beimplemented as a system, a method, an integrated circuit, a computerprogram, a recording medium such as a computer-readable compactdisc-read-only memory (CD-ROM), or any selective combination thereof.

Embodiments will be described below with reference to the drawings.

All embodiments described below illustrate comprehensive or specificexamples. Numerals, shapes, materials, constituent elements, theplacement positions and connection forms of these constituent elements,steps, the sequence of these steps, and the like are only examples, andare not intended to restrict the present disclosure. Of the constituentelements described in the embodiments below, constituent elements notdescribed in independent claims, each of which indicates the topmostconcept, will be described as optional constituent elements.

First Embodiment

In this embodiment, a head-mounted display apparatus and the like thatappropriately present information to a user as desired by the user willbe described. More specifically, a head-mounted display apparatus andthe like will be described that continue to display information that issuch that it takes a relative long time for the user to grasp theinformation according to the user's manipulation.

FIG. 1 illustrates an outside shape of a head-mounted display apparatus10 in the first embodiment.

The head-mounted display apparatus 10 is an eyeglass-type terminal thatis worn on the head of a user as illustrated in FIG. 1. The head-mounteddisplay apparatus 10 has a main body 11, nose pads 12, temples 13, andearpieces 14.

The main body 11 is a processor that causes the head-mounted displayapparatus 10 to perform information processing, display an image, andperform other processing. The main body 11 is attached to part of thetemples 13. When the position or attitude of the user's head is changed,the position or attitude of the main body 11 is changed together withthe user's head. The functions of the main body 11 and processing by itwill be described later in detail.

The nose pads 12 fix the head-mounted display apparatus 10 to the user'shead. To fix the head-mounted display apparatus 10 to the user's head,the nose pads 12 are placed so as to sandwich the user's nose from boththe right side and the left side.

Each temple 13 is a rod that links the relevant nose pad 12 and earpiece14 together.

The earpieces 14 fix the head-mounted display apparatus 10 to the user'shead. To fix the head-mounted display apparatus 10 to the user's head,the earpieces 14 are placed so as to rest on the user's ears.

The depth direction when viewed by the user wearing the head-mounteddisplay apparatus 10 will also be referred to as the X direction, theleft and right direction will also be referred to as the Y direction,and the direction in which the top of the head and feet are mutuallylinked will also be referred to as the Z direction. Unless otherwisenoted, the positive direction of the Z direction will be referred to asthe upward direction and its negative direction will be referred to asthe downward direction.

The shape of the head-mounted display apparatus 10 illustrated in FIG. 1is just an example. If the head-mounted display apparatus 10 is worn onat least part of the user's head and displays an image to be presentedto the user, the head-mounted display apparatus 10 may have any shape.For example, the head-mounted display apparatus 10 may be of a cap typeor a helmet type.

FIG. 2 is a block diagram illustrating the hardware structure of thehead-mounted display apparatus 10 in this embodiment.

As a hardware structure, the head-mounted display apparatus 10 has acentral processing unit (CPU) 101, a main memory 102, a storage 103, acommunication interface 104, an acceleration sensor 105, a button 106,and a display 107, as illustrated in FIG. 2.

The CPU 101 a processor that executes a control program stored in thestorage 103 or the like.

The main memory 102 is a volatile storage area used by the CPU 101 as awork area when the CPU 101 executes the control program.

The storage 103 is a non-volatile storage area that retains the controlprogram and content.

The communication interface 104 is a network interface that transmitsdata to and receives data from another apparatus through a network. Thecommunication interface 104 is, for example, a wireless local areanetwork (LAN) interface conforming to the IEEE802.11a/b/g standard.

The acceleration sensor 105 is a sensor that detects acceleration.

The button 106 (also referred to as the manipulation button) is an inputdevice that accepts a manipulation performed by the user such aspressing or a touch. A touchpad (not illustrated) can also be usedinstead of the button 106. In addition, the button 106 can be replacedwith a combination of a microphone (not illustrated) that collectssurrounding voice and a voice recognizing apparatus (not illustrated)that acquires a speech of the user through voice recognition of thevoice collected by the microphone. In this case, the acquisition of acertain speech by the voice recognizing apparatus is accepted as theabove manipulation.

The display 107 is a transparent display device that displays an image.The display 107 is, for example, a prism display that presents an imageto the user by using a half mirror. As long as the display 107 presentsan image to the user, the type of the display 107 is not important. Thedisplay 107 may be, for example, a so-called retinal scanning displaythat uses laser beams to present an image to the retinas of the user.

FIG. 3 is a block diagram illustrating the functional structure of thehead-mounted display apparatus 10 in this embodiment.

As a functional structure, the head-mounted display apparatus 10 has anattitude detector 111, a manipulation detector 112, a controller 113,and a displayer 114 as illustrated in FIG. 3. The controller 113 mayhave a decider 115 as its one function.

The attitude detector 111 is a processor that detects an attitude of thehead-mounted display apparatus 10. Specifically, the attitude detector111 causes the acceleration sensor 105 to detect an attitude of thehead-mounted display apparatus 10 and a change in the attitude andcreates attitude information, which indicates the change in theattitude. The attitude of the head-mounted display apparatus 10 has aone-to-one correspondence with the orientation of the user's head. If,for example, the user's head is in the front-upward attitude at anangle, the attitude detector 111 creates attitude information indicating“upward inclination” (simply referred to below as “upward”). Theattitude detector 111 is implemented by the CPU 101, the main memory102, the storage 103, the acceleration sensor 105, and the like.

A change in the attitude of the head-mounted display apparatus 10 iscaused by either of a change in the orientation of the whole of theuser's body and a change in the motion of the user's neck. These changesare identified by the value of acceleration detected by the accelerationsensor 105. Specifically, if the value of acceleration detected by theacceleration sensor 105 is equal to or larger than a predeterminedvalue, the change is identified as having been caused by the motion ofthe neck. The attitude detector 111 may create information indicatingthe above attitude only when the attitude detector 111 detects a changein the attitude caused by the motion of the user's neck. This is becausea change in the attitude caused by the motion of the user's neck isconsidered to more clearly indicate that the user has intentionallychanged the attitude of the head.

The attitude detector 111 may detect five directions, which are upward,downward, leftward, rightward, and frontward directions, as attitudes.The user's head orientation intended by the user may not exactly matchthe actual user's head orientation. For example, even if the userintends to move the user's head upward, the actual user's headorientation may not be in the accurate upward direction; the user's headmay be oriented in a direction that is slightly inclined to the left (orright) with respect to the upward direction. In view of this, when theattitude detector 111 is structured so that it detects the above fivedirections as attitudes, it is possible to determine a direction, asdescribed above, that is slightly deviated from the accurate upwarddirection as the upward direction. When not only the exact upwarddirection but also a direction within a predetermined range includingthe exact upward direction are determined as the upward direction,flexible manipulations by the user are accepted.

The manipulation detector 112 is a processor that detects a manipulationperformed on the head-mounted display apparatus 10 by the user. Themanipulation detector 112 detects the pressing of the button 106 by theuser or a touch to it by the user. The manipulation detector 112 isimplemented by the CPU 101, the main memory 102, the storage 103, thebutton 106, and the like.

The controller 113 is a processor that switches information to bedisplayed by the displayer 114 according to a change, detected by theattitude detector 111, in the attitude of the head-mounted displayapparatus 10 (the information will also be referred to below as thedisplay information). In addition, when the manipulation detector 112detects a manipulation, even if there is a change, detected by theattitude detector 111, in the attitude of the head-mounted displayapparatus 10, the controller 113 maintains the display information to bedisplayed by the displayer 114 without switching the displayinformation.

Specifically, the controller 113 causes the displayer 114 to displayvarious types of display information. More specifically, the controller113 holds a correspondence in advance between each of the attitudes ofthe head-mounted display apparatus 10 and display information to bedisplayed by the displayer 114 as commanded by the controller 113 whenthe head-mounted display apparatus 10 is in the attitude. The controller113 acquires attitude information from the attitude detector 111,selects display information associated with the acquired attitudeinformation, and causes the displayer 114 to display the selecteddisplay information. The display information may be information that isobtained when the controller 113 executes a predetermined application tooutput the display information. Alternatively, the display informationmay be predetermined information or may be information obtained from theoutside through the communication interface 104.

The controller 113 displays icons at portions, on the display area ofthe displayer 114, in the upward, downward, leftward, and rightwarddirections, each icon indicating one of the four information items thatis associated with the direction. These icons help the user know thatwhat information is displayed by inclining the head-mounted displayapparatus 10 in what direction.

When maintaining the display information, the controller 113 maymaintain the type of the display information. Specifically, thecontroller 113 may maintain the type of the display information and mayhandle the display information itself so that it is switched within therange of the type.

The controller 113 may change the display form of display information tobe presented to the user when the manipulation detector 112 detects amanipulation to notify the user that display information is beingmaintained by the displayer 114. Specifically, the controller 113 maychange the display form of display information so that it ishighlighted. Accordingly, it is possible to have the user explicitlyrecognize that even if the attitude of the head-mounted displayapparatus 10 is changed, the display information is maintained withoutbeing switched.

As described above, the controller 113 may have the decider 115. In thiscase, the controller 113 switches or maintains information to bedisplayed on the displayer 114 according not only to the attitude of thehead-mounted display apparatus 10 but also to the result of a decisionmade by the decider 115 about display information.

The decider 115 is a processor that decides whether the amount ofinformation to be displayed by the displayer 114 is larger than orsmaller than a predetermined value. The amount of information is anamount decided according to a display area occupied by information to bedisplayed by the displayer 114, the number of characters, and thepresence or absence of a mixture of images and characters. Informationis decided to have a large amount of information when, for example, theinformation occupies a display area larger than the predetermined value,the information occupies a display area the ratio of which to thedisplayable area of the display 107 is larger than a predeterminedvalue, the information includes more characters or images than apredetermined value, or the information includes both images andcharacters. Since it is possible to infer to a certain extent whetherthe user intends to switch the display information according to thisamount of information, the controller 113 can decide whether to switchor maintain the display information according to the result of the abovedecision. The decider 115 is equivalent to a first decider or a seconddecider in WHAT IS CLAIMED IS.

The displayer 114 is a display device that displays information as animage. Specifically, the displayer 114 displays display informationselected by the controller 113 as an image. The displayer 114 isimplemented by the display 107 and the like.

The functions of the functional blocks of the head-mounted displayapparatus 10 and processing by these functional blocks will be describedbelow in detail. First, how the attitude detector 111 detects theattitude of the head-mounted display apparatus 10 will be described.

FIG. 4 illustrates how the attitude detector 111 in this embodimentdetects an attitude. FIG. 4 illustrates a change in the attitude of thehead-mounted display apparatus 10 when the user shakes the neck in ahorizontal plane (XY plane), that is, when the user rotates the headhorizontally (to the left or right).

Specifically, FIG. 4 illustrates a case in which the user rotates thehead so the head is oriented to a direction inclined through θ1 degreesto the left with respect to the direction in which the user views thefront. At that time, the attitude detector 111 detects from theacceleration sensor 105 that the head-mounted display apparatus 10 haschanged from the reference attitude to an attitude rotated through θ1degrees to the left with respect to the reference attitude. Thereference attitude is an attitude at a time when this attitudes startsto change, that is, the reference attitude is an attitude in a case inwhich the user's head is thought to be substantially stationary.

FIG. 5 also illustrates how the attitude detector 111 in this embodimentdetects an attitude. FIG. 5 illustrates a change in the attitude of thehead-mounted display apparatus 10 when the user shakes the neck in thevertical direction (in an XZ plane), that is, when the user shakes thehead vertically (upward or downward).

Specifically, FIG. 5 illustrates a case in which the user shakes thehead so the head is oriented to a direction inclined through θ2 degreesupward with respect to the direction in which the user views the front.At that time, the attitude detector 111 detects from the accelerationsensor 105 that the head-mounted display apparatus 10 has changed fromthe reference attitude to an attitude inclined through θ2 degrees upwardwith respect to the reference attitude.

The attitude detector 111 detects the attitude of the head-mounteddisplay apparatus 10 according to θ1 and θ2 obtained as described above.The attitude of the head-mounted display apparatus 10 may be identifiedas one of an unlimited number of steps through a combination of θ1 andθ2. Alternatively, the attitude may be identified as a rough attitudesuch as, for example, an upward attitude or downward attitude bydividing the range of θ1 and θ2 into a predetermined number of segments.In the description below, the latter case will be described;specifically, a case will be described in which the attitude of thehead-mounted display apparatus 10 is identified, according to θ1 and θ2,as one of five attitudes, which are an upward attitude, a downwardattitude, a leftward attitude, a rightward attitude, and a frontwardattitude.

FIG. 6 illustrates a table 601, in this embodiment, that indicatesattitude information the controller 113 and types of display informationdisplayed by the controller 113 in correspondence to each other.Specifically, the table 601 indicates correspondence between eachattitude of the head-mounted display apparatus 10 and a type of displayinformation to be displayed by the controller 113 on the displayer 114when the attitude is taken.

For example, the upward attitude of the head-mounted display apparatus10 is associated with “weather forecast” which is one type of thedisplay information. That is, if the attitude detector 111 detects thatthe attribute of the head-mounted display apparatus 10 is the upwardattitude, the controller 113 displays information about the weatherforecast on the displayer 114. To display information about a weatherforecast on the displayer 114, the controller 113 may obtain in advancenecessary information from the outside through the communicationinterface 104 or the like by using a predetermined application or thelike and may create display information by performing informationprocessing on the obtained information.

The front attitude of the head-mounted display apparatus 10 isassociated with an indication that there is no relevant displayinformation. This means that when the user faces the front, there is nodisplay information, that is, the displayer 114 displays no information.

Display information that the displayer 114 displays will be describedbelow.

FIG. 7 illustrates, in this embodiment, the field of view 701 of theuser and icons displayed on the displayer 114.

The field of view 701 of the user indicates an extent viewable to theuser wearing the head-mounted display apparatus 10. The field of view701 of the user includes an external landscape and also includes thedisplayer 114 so that it is superimposed on the external landscape. Thatis, the user views the external landscape through the displayer 114. Theicons 711, 712, 713, and 714 are displayed at portions, on the displayarea of the displayer 114, that indicate the upward, downward, leftward,and rightward directions, respectively. These icons present to the useran attitude of the head-mounted display apparatus 10 (a direction inwhich the head-mounted display apparatus 10 is inclined) and a type ofdisplay information in the attitude. These icons help the user know thatwhen the user changes the attitude of the head-mounted display apparatus10 to the direction indicated by an icon, the display informationindicated by the icon is displayed.

As described above, the user uses the head-mounted display apparatus 10to visually recognize an external landscape and information superimposedon the external landscape.

FIG. 8 illustrates, in this embodiment, the field of view of the userand an image 801 displayed on the displayer 114. The image 801 is anexample of an image displayed by the displayer 114 according to a changein the attitude of the head-mounted display apparatus 10 as commanded bythe controller 113.

The image 801 indicates a weather forecast image, which is informationdisplayed when the attitude of the head-mounted display apparatus 10 isin the upward direction. The image 801, which is enclosed by a frame802, includes text and a figure that indicate the weather of the day.

In this case, like the image 801, a white color may be displayed on aportion that becomes the background of the information indicating theforecasted weather. Alternatively, nothing may be displayed so that theuser can visually recognize the external landscape through the displayer114.

FIG. 9 illustrates a highlighted display form in this embodiment. Animage 901 is an example of a display image on which the controller 113has changed the display form in response to a predetermined manipulationperformed by the user.

Specifically, on the image 901, the frame 902 of the image has beenchanged so as to be thicker than on the image 801. Thus, thehead-mounted display apparatus 10 enables the user to aggressivelyrecognize that the display information is being maintained on thedisplayer 114, that is, the display is not switched even in a case inwhich the attitude is changed.

To indicate that the display information is being maintained on thedisplayer 114, the image may be highlighted in any other method insteadof changing the frame of the image to a thick frame. Specifically, thecolor of a figure used as the background of the image may be changed,the color or ornament (such as a font, a bold style, an italic style, orunderlines) of text may be changed, or a plurality of methods describedabove may be combined.

Processing to change information to be displayed by the 114 as commandedby the controller 113 will be described below.

FIG. 10 is a flowchart of processing performed by the controller 113 inthis embodiment to switch or maintain display information. FIGS. 12A,12B, 12C, 12D and 12E schematically illustrates a series of flow inwhich the controller 113 in this embodiment maintains a display andterminates the maintaining of the display.

In step S101, it is decided whether the attitude detector 111 hasdetected a change in the attitude of the head-mounted display apparatus10. If the attitude detector 111 has detected a change in the attitude(the result in step S101 is Yes), the attitude detector 111 createsattitude information indicating the detected change in the attitude,after which the sequence proceeds to step S102. If the attitude detector111 has not detected a change in the attitude (the result in step S101is No), step S101 is executed again. That is, the attitude detector 111is placed on standby in step S101 until it detects a change in theattitude. Step S101 is equivalent to the state in FIG. 12A.

In step S102, the controller 113 acquires the attitude informationcreated by the attitude detector 111 in step S101, and causes thedisplayer 114 to display the display information associated with theattitude of the head-mounted display apparatus 10, the attitude beingindicated in the attitude information. Step S101 is equivalent to thestate in FIG. 12B. In FIG. 12B, information about a weather forecast isdisplayed on the displayer 114 because the attitude of the head-mounteddisplay apparatus 10 has been changed to the upward attitude.

In step S103, it is decided whether the manipulation detector 112 hasdetected a predetermined manipulation performed by the user. If themanipulation detector 112 has detected a predetermined manipulationperformed by the user (the result in step S103 is Yes), the sequenceproceeds to step S111. If the manipulation detector 112 has not detecteda predetermined manipulation performed by the user (the result in stepS103 is No), the sequence proceeds to step S104. Step S103 is equivalentto the state in FIG. 12C. In FIG. 12C, the display of the informationabout a weather forecast on the displayer 114 is maintained because apredetermined manipulation performed by the user (for example, thepressing of the button 106 by a finger U1 of the user) has beendetected. At that time, the controller 113 changes the display form bychanging the frame of the display of the information to a thick framewhile maintaining the display information without switching it.

In step S104, the attitude detector 111 is placed on standby in stepS104 until it detects a change in the attitude of the head-mounteddisplay apparatus 10. Upon the detection of a change in the attitude,the attitude detector 111 creates attitude information, after which thesequence proceeds to step S105.

In step S105, the controller 113 acquires the attitude informationcreated by the attitude detector 111 in step S104 and switchesinformation to be displayed by the displayer 114 according to theattitude information.

In step S111, the attitude detector 111 is placed on standby in stepS111 until it detects a change in the attitude of the head-mounteddisplay apparatus 10. Upon the detection of a change in the attitude,the attitude detector 111 creates attitude information, after which thesequence proceeds to step S112.

In step S112, the controller 113 maintains the display information onthe displayer 114 without switching it, regardless of the attitudeinformation created by the attitude detector 111 in step S111. Step S112is equivalent to the state in FIG. 12D. The weather forecast informationis displayed on the displayer 114 in a state in which the information isenclosed by a thick frame, in spite of the head-mounted displayapparatus 10 facing the front.

The order of the detection of a predetermined manipulation (step S103)and the detection of a change in the attitude (step S104 or S111) may beopposite to the order illustrated in FIG. 10.

In a series of processing described above, if the user performs apredetermined manipulation, the head-mounted display apparatus 10maintains information to be presented to the user without switching it(see FIG. 12D). Thus, the head-mounted display apparatus 10 canappropriately present information to the user as desired by the user.

FIG. 11 is a flowchart of processing performed by the controller 113 inthis embodiment to terminate the maintaining of a display.

In step S201, it is decided whether the manipulation detector 112 hasdetected a predetermined manipulation performed by the user (forexample, the pressing of the button 106 by the finger U1 of the user,which is also referred to as a subsequent manipulation). If themanipulation detector 112 has detected a predetermined manipulationperformed by the user (the result in step S201 is Yes), the sequenceproceeds to step S202. If the manipulation detector 112 has not detecteda predetermined manipulation performed by the user (the result in stepS201 is No), the sequence proceeds to step S211.

In step S202, the controller 113 terminates the maintaining of theinformation to be displayed by the displayer 114. That is, thecontroller 113 makes the information to be displayed by the displayer114 ready for being switched. At that time, the controller 113 mayswitch the display information to be displayed on the displayer 114 ormay delete the display on the displayer 114 (see FIG. 12E) according tothe attitude detected by the attitude detector 111.

In step S211, the controller 113 continues to maintain information to bedisplayed by the displayer 114.

Thus, the head-mounted display apparatus 10 can appropriately terminatethe maintaining of the display information to be displayed by thedisplayer 114.

The controller 113 may perform the above processing on all informationto be displayed by the displayer 114 in a consistent manner or maydecide for each piece of information whether to perform the aboveprocessing according to the amount of display information, which will bespecifically described below by using two examples of images. In a firstexample, the number of images included in display information is used asthe amount of information. In a second example, the number of charactersincluded in display information is used as the amount of information.

FIG. 13 illustrates display information, in the first embodiment, thatincludes a large number of images. A first exemplary method will bedescribed with reference to FIG. 13, which is a method of decidingwhether to perform the above processing according to the number ofimages included in display information.

In FIG. 13, an image 1301 to be displayed by the displayer 114 indicatesweather in one week. The image 1301 is an example of an image thatincludes much more information than the image 801 that indicates theweather of a day. It can be thought that it takes a longer time for theuser to grasp information indicated on the image 1301 when compared withthe image 801. By contrast, it can be thought that it takes a short time(several seconds, for example) for the user to grasp information on theimage 801 that indicates the weather of a day.

Therefore, the controller 113 may decide whether to perform the seriesof processing illustrated in FIG. 10 according to the result of adecision made by the decider 115.

If the number of images included on the image to be displayed on thedisplayer 114 exceeds, for example, 3, the decider 115 decides that theimage includes much information. The controller 113 performs the seriesof processing in FIG. 10 only when the amount of information on theimage is large. If the decider 115 decides that the amount ofinformation on the image is small, the controller 113 performsprocessing to switch the display information according to the detectedchange in attitude.

In this decision method, when the decider 115 decides that the number ofimages included on the image 801 is 1 and the controller 113 displaysthe image 801, the controller 113 performs processing to switch thedisplay information according to the detected change in attitude. Whenthe decider 115 decides that the number of images included on the image801 is 7 and the controller 113 displays the image 1301, the controller113 performs processing to maintain the display information according toa predetermined manipulation (processing in FIG. 10).

FIG. 14 illustrates display information, in this embodiment, thatincludes a small number of characters. FIG. 15 illustrates displayinformation, in this embodiment, that includes a large number ofcharacters. A second exemplary method will be described with referenceto FIGS. 14 and 15, which is a method of deciding whether to perform theabove processing according to the number of characters included indisplay information.

An image 1401 illustrated in FIG. 14 includes a relatively small numberof characters (about 15 characters, for example). An image 1501illustrated in FIG. 15 includes a relatively large number of characters(about 100 characters, for example).

If the number of characters included on an image to be displayed by thedisplayer 114 exceeds, for example, 20, the decider 115 decides that theamount of information is large. Only in this case, the controller 113performs the series of processing illustrated in FIG. 10. Otherwise, thecontroller 113 performs processing to switch the display informationaccording to the detected change in attitude.

In the decision of the number of characters, the decider 115 may decidethe number of characters excluding fixed character strings in thedisplay information. If, for example, the display information is mail,the decider 115 may exclude “From” indicating a transmission source or“To” indicating a transmission destination and may count the number ofremaining characters. This is because these fixed character strings donot have a meaning specific to the display information, so it cannot bethought that the user strives to understand the meanings of the fixedcharacter strings.

When the decider 115 makes a decision about the number of charactersincluded in one line, the decider 115 may further consider the number ofcharacters per line, the number of lines, a character display size, andthe like. Specifically, for example, many characters are included in oneline, there are many lines, or the character display size is small, thedecider 115 may decide that the amount of information is large.

Therefore, if an image is such that it takes a relative long time forthe user to grasp the image, the user can maintain the displayinformation by performing a predetermined manipulation on thehead-mounted display apparatus 10. If an image is such that it does nottake so long time for the user to grasp the image, the user can switchthe display information according to the attitude. The user can use thehead-mounted display apparatus 10 to reliably obtain information withoutlosing the convenience of switching display information.

As described above, the head-mounted display apparatus in thisembodiment can appropriately present information to the user as desiredby the user. In particular, when the user fixes (maintains) the displayof information by performing a predetermined manipulation on the basisof the user's own decision, the head-mounted display apparatus candisplay necessary information regardless of the attitude of thehead-mounted display apparatus. Thus, the user not only can collectinformation through an intuitive natural motion without making personsaround the user feel uncomfortable but also can grasp information thatis such that it takes a relative long time for the user to grasp theinformation.

Second Embodiment

In this embodiment, a head-mounted display apparatus and the like thatappropriately present information to a user as desired by the user willbe described. More specifically, a head-mounted display apparatus andthe like will be described that display information desired by the useraccording to the user's manipulation.

FIG. 16 is an outside shape of a head-mounted display apparatus 10A inthis embodiment.

Although, as illustrated in FIG. 16, the head-mounted display apparatus10A has an outside shape similar to the outside shape of thehead-mounted display apparatus 10 in the first embodiment, thehead-mounted display apparatus 10A differs from the head-mounted displayapparatus 10 in that the head-mounted display apparatus 10A has a mainbody 11A. This difference will be described below as a difference in thehardware structure.

FIG. 17 is a block diagram illustrating the hardware structure of thehead-mounted display apparatus 10A in this embodiment.

Although, as illustrated in FIG. 17, the head-mounted display apparatus10A has a hardware structure similar to the hardware structure of thehead-mounted display apparatus 10 in the first embodiment, thehead-mounted display apparatus 10A differs from the head-mounted displayapparatus 10 in that the head-mounted display apparatus 10A has a camera108. In other respects, the structure is the same as in the firstembodiment, so their descriptions will be omitted.

The camera 108 is an imaging apparatus that creates an image bycapturing a surrounding image. The camera 108 is disposed on thehead-mounted display apparatus 10A at a position and an angle at whichthe camera 108 can capture an image in the forward direction (positivedirection of the X direction). The camera 108 may be a visible lightcamera that captures a surrounding image with visible light or aninfrared camera that captures a surrounding mage with infrared rays.

FIG. 18 is a block diagram illustrating the functional structure of thehead-mounted display apparatus 10A in the second embodiment.

Although, as illustrated in FIG. 18, the head-mounted display apparatus10A in this embodiment has a functional structure similar to thefunctional structure of the head-mounted display apparatus 10, thehead-mounted display apparatus 10A differs from the head-mounted displayapparatus 10 in that the head-mounted display apparatus 10A has agesture detector 121. In other respects, the structure is the same as inthe first embodiment, so their descriptions will be omitted.

The gesture detector 121 is a processor that detects a swipe gesturemanipulation performed by the user in a manipulation space, the wipegesture manipulation being a gesture manipulation to indicate onedirection. The swipe gesture manipulation is the user's motion in whichthe user moves a hand or the like in one direction like a sweep motion.The swipe gesture manipulation may be simply referred to as the gesturemanipulation.

Specifically, the gesture detector 121 captures an image of amanipulation space ahead of the head-mounted display apparatus 10A withthe camera 108. In the capturing of the manipulation space, a pluralityof still images (frames) are created; a moving picture is taken or astill picture is taken a plurality of times. The manipulation space is aspace in which the user performs a swipe gesture manipulation; themanipulation space occupies part or the whole of a space within whichthe camera 108 can capture an image.

The gesture detector 121 detects a motion of the hand of the user fromthe frames obtained by photography and creates gesture information,which indicates the motion of the hand. If the camera 108 is a visiblelight camera, it extracts, for example, skin-color portions on the imageto distinguish the user's hand from other subjects, after which thecamera 108 detects the motion of the user's hand and creates gestureinformation. If the camera 108 is an infrared camera, it detects themotion of the user's hand by detecting infrared rays released from theuser's hand and creates gesture information. The gesture manipulation isnot limited to a motion of a hand; the gesture manipulation may be amotion of another portion of the user's body. The gesture detector 121is implemented by the CPU 101, the main memory 102, the storage 103, thecamera 108, and the like.

The gesture detector 121 may detect swipe gesture manipulations only infour directions, which are the upward, downward, leftward, and rightwarddirections, when viewed from the user. It can be thought that adirection intended by the user's swipe gesture manipulation does notexactly match the direction in which the user actually moves the user'shand so as to sweep something. For example, even if the user intends toperform an upward swipe gesture manipulation, the direction in which theuser actually moves the user's hand may not be the accurate upwarddirection; the actual direction may be oriented in a direction that isslightly inclined with respect to the upward direction. If the gesturedetector 121 is structured so that it detects swipe gesturemanipulations only in the above four directions, it is possible todetermine a direction, as described above, that is slightly deviatedfrom the accurate upward direction as the upward direction.

The controller 113 displays icons at portions, on the display area ofthe displayer 114, in the upward, downward, leftward, and rightwarddirections, each icon indicating one of the four information items thatis associated with the direction, as in the first embodiment. Theseicons help the user know that what information is displayed byperforming a swipe gesture manipulation in what direction.

FIG. 19 is a flowchart of processing performed by the controller 113 inthis embodiment to switch a display. FIGS. 20A, 20B and 20Cschematically illustrate a series of flow in which the controller 113 inthe second embodiment maintains a display and terminates the maintainingof the display. A series of flow in which the controller 113 maintains adisplay and terminates the maintaining of the display will be describedwith reference to FIGS. 19, 20A, 20B and 20C.

First, it is assumed that the attitude of the head-mounted displayapparatus 10A is oriented to the front and that nothing is displayed onthe displayer 114. This state is equivalent to the state in FIG. 20A.

In step S301, it is decided whether the gesture detector 121 hasdetected a swipe gesture manipulation performed by the user. If thegesture detector 121 has detected a swipe gesture manipulation (theresult in step S301 is Yes), the gesture detector 121 creates gestureinformation, after which the sequence proceeds to step S311. If thegesture detector 121 has not detected a swipe gesture manipulation (theresult in step S301 is No), the sequence proceeds to step S302. StepS301 is equivalent to the state in FIG. 20B. In FIG. 20B, informationabout a weather forecast is displayed on the displayer 114 in responseto a swipe gesture manipulation performed by the user from above to down(downward manipulation) with a hand U2, the weather forecast beingassociated with the upward direction, which corresponds to an upperportion that is the start point of the swipe gesture manipulation.

In step S302, it is decided whether the attitude detector 111 hasdetected a change in the attitude of the head-mounted display apparatus10A. If the attitude detector 111 has detected a change in the attitude,the attitude detector 111 creates attitude information, after which thesequence proceeds to step S303.

In step S303, the controller 113 switches information to be displayed bythe displayer 114 according to the attitude information created by theattitude detector 111 in step S302.

In step S311, the controller 113 acquires the gesture informationcreated by the gesture detector 121 in step S301 and switchesinformation to be displayed by the displayer 114 according to thegesture information.

It is also possible to delete the display (terminate the displaying ofinformation) by performing a swipe gesture manipulation indicating adirection opposite to the direction of the swipe gesture manipulationperformed to display information on the displayer 114 (see FIG. 20C).

The head-mounted display apparatus 10A can be combined with thehead-mounted display apparatus 10 in the first embodiment. Specifically,when the head-mounted display apparatus 10A further has the button 106and manipulation detector 112, a display can be maintained according toa predetermined manipulation performed by the user.

As described above, the head-mounted display apparatus in thisembodiment can appropriately present information to the user as desiredby the user. In particular, it is possible to switch display informationby performing a swipe manipulation instead of changing the attitude ofthe head-mounted display apparatus. Therefore, while, for example,facing the front, the user can cause the head-mounted display apparatusto display necessary information by performing a swipe manipulation andcan grasp the information; the necessary information would otherwise bedisplayed by changing the attitude of the head-mounted displayapparatus. Thus, the user not only can collect information through anintuitive natural motion without making persons around the user feeluncomfortable but also can grasp information that is such that it takesa relative long time for the user to grasp the information.

In the above embodiments, each constituent element may be structured byusing special hardware or may be implemented by executing a softwareprogram suitable to the constituent element. Alternatively, to implementeach constituent element, a CPU, a processor, or another programexecutor may read out and execute a software program recorded in a harddisk, a semiconductor memory, or another recording medium. Software thatimplements the head-mounted display apparatuses and the like in theabove embodiments is a program described below.

That is, this program causes a computer to execute a control method ofcontrolling a head-mounted display apparatus worn on the head of a user;the head-mounted display apparatus has a displayer on which informationis displayed; the control method includes an attitude detecting step ofdetecting an attitude of the head-mounted display apparatus, amanipulation detecting step of detecting a predetermined manipulationperformed on the head-mounted display apparatus, and a control step ofswitching information to be displayed by the displayer according to achange in the attitude detected in the attitude detecting step; in thecontrol step, if the predetermined manipulation is detected in themanipulation detecting step, even if there is a change in the attitudedetected in the attitude detecting step, the information being displayedby the displayer is maintained without being switched.

This program causes a computer to execute a control method ofcontrolling a head-mounted display apparatus worn on the head of a user;the head-mounted display apparatus has a displayer on which informationis displayed; the control method includes an attitude detecting step ofdetecting an attitude of the head-mounted display apparatus, a gesturedetecting step of detecting a swipe gesture manipulation that indicatesone direction, the swipe gesture manipulation being performed in amanipulation space, and a control step of switching information to bedisplayed by the displayer according to a change in the attitudedetected in the attitude detecting step; in the control step, theinformation to be displayed by the displayer is switched according tothe direction indicated by the swipe gesture manipulation.

So far, the head-mounted display apparatus and the like in one or aplurality of aspects have been described according to the embodiments.However, the present disclosure is not limited to these embodiments. Therange of one or a plurality of aspects may include embodiments in whichvarious variations that a person having ordinary skill in the art thinksof are applied to the embodiments described above and may also includeembodiments in which constituent elements in different embodiments arecombined, without departing from the intended scope of the presentdisclosure.

The present disclosure can be applied to a head-mounted displayapparatus that appropriately presents information to a user as desiredby the user.

The invention claimed is:
 1. A head-mounted display apparatus worn on ahead of a user, the apparatus comprising: an attitude sensor thatdetects an attitude of the head-mounted display apparatus; amanipulation sensor that detects a predetermined manipulation performedon the head-mounted display apparatus; a display; a processor; and amemory having a program stored in the memory, the program causing theprocessor to execute operations in which first display informationdisplayed on the display is switched to second display informationaccording to a change in the attitude detected by the attitude sensor,and when the manipulation sensor detects the predetermined manipulation,the first display information being displayed on the display ismaintained without being switched to the second display information evenif there is a change in the attitude detected by the attitude sensor. 2.The head-mounted display apparatus according to claim 1, wherein, whenthe manipulation sensor detects the predetermined manipulation and thechange in the attitude is detected by the attitude sensor, the firstdisplay information being displayed on the display is maintained withoutbeing switched to the second display information and a display form ofthe first display information is changed.
 3. The head-mounted displayapparatus according to claim 2, wherein when the first displayinformation is displayed on the display, a figure is displayed as abackground of the first display information together with the firstdisplay information; and the display form of the first displayinformation is changed by changing a color of the figure.
 4. Thehead-mounted display apparatus according to claim 2, wherein when thefirst display information is displayed on the display, a frame thatencloses the first display information is displayed together with thefirst display information; and the display form of the first displayinformation is changed by changing a thickness or a color of the frame.5. The head-mounted display apparatus according to claim 1, wherein theoperations further comprise: determining, by the processor, whether anamount of information to be displayed on the display is larger than apredetermined value; and if the amount of information is determined tobe larger than the predetermined value, and when the manipulation sensordetects the predetermined manipulation, the first display informationbeing displayed on the display is maintained without being switched tothe second display information even if there is a change in the attitudedetected by the attitude sensor.
 6. The head-mounted display apparatusaccording to claim 1, wherein the operations further comprise:determining, by the processor, whether the amount of information to bedisplayed on the display is smaller than a predetermined value; and ifthe amount of information is determined to be smaller than thepredetermined value, the first display information displayed on thedisplay is switched to the second display information according to thechange in the attitude detected by the attitude sensor even if themanipulation sensor detects the predetermined manipulation.
 7. Thehead-mounted display apparatus according to claim 1, wherein, if themanipulation sensor detects the predetermined manipulation and furtherdetects a subsequent manipulation that is same as the predeterminedmanipulation, the first display information displayed on the display isswitched to the second display information according to the attitudedetected by the attitude sensor when the manipulation sensor detects thesubsequent manipulation.
 8. The head-mounted display apparatus accordingto claim 1, wherein, if the manipulation sensor detects thepredetermined manipulation and further detects a subsequent manipulationthat is same as the predetermined manipulation, display of the firstdisplay information on the display is stopped.
 9. The head-mounteddisplay apparatus according to claim 1, wherein: the manipulation sensorhas a manipulation button; and the manipulation sensor detects, as thepredetermined manipulation, pressing of the button or a touch to thebutton.
 10. The head-mounted display apparatus according to claim 1,wherein: the manipulation sensor has a touchpad; and the manipulationsensor detects a tap on the touchpad as the predetermined manipulation.11. The head-mounted display apparatus according to claim 1, wherein:the manipulation sensor has a microphone and a voice recognizingapparatus that acquires a speech of the user through voice recognitionof voice collected by the microphone; and the manipulation sensordetects, as the predetermined manipulation, acquisition of apredetermined speech by the voice recognizing apparatus.
 12. Thehead-mounted display apparatus according to claim 1, wherein: theattitude sensor generates attitude information, which indicates adirection to which the attitude has been changed from a referenceattitude, the direction being one of an upward direction, a downwarddirection, a leftward direction, and a rightward direction when viewedfrom the user; and in the operations, four information items associatedwith the upward direction, downward direction, leftward direction, andrightward direction, respectively, are stored in the memory asinformation to be displayed on the display, icons are displayed atportions, on a display area of the display, in the upward direction,downward direction, leftward direction, and rightward direction, each ofthe icons indicating one of the four information items, and when theattitude sensor generates the attitude information, informationassociated with a direction indicated by the attitude information isdisplayed on the display.
 13. The head-mounted display apparatusaccording to claim 1, further comprising a gesture sensor that detects aswipe gesture manipulation that indicates one direction, the swipegesture manipulation being performed in a manipulation space, wherein:the first display information displayed on the display is switched tothe second display information according to the direction indicated bythe swipe gesture manipulation.
 14. The head-mounted display apparatusaccording to claim 13, wherein: the gesture sensor detects a motion of ahand of the user moving in a downward motion as a swipe gesturemanipulation indicating a downward direction, detects a motion of thehand of the user moving in a upward motion as a swipe gesturemanipulation indicating an upward direction, detects a motion of thehand of the user moving in a rightward direction as a swipe gesturemanipulation indicating a rightward direction, and detects a motion ofthe hand of the user moving in a leftward direction as a swipe gesturemanipulation indicating a leftward direction; and the first displayinformation displayed on the display is switched to the second displayinformation according to the direction indicated by the swipe gesturemanipulation detected by the gesture sensor.
 15. The head-mounteddisplay apparatus according to claim 13, wherein: the gesture sensor hasa camera that captures an image of the hand of the user with visiblelight; and the gesture sensor detects a swipe gesture manipulationaccording to a motion of the hand of the user, the motion of the hand ofthe user being determined based on the image of the hand captured by thecamera.
 16. The head-mounted display apparatus according to claim 13,wherein: the gesture sensor has an infrared sensor that detects aninfrared ray released from the hand of the user; and the gesture sensordetects a swipe gesture manipulation according to a motion of the handof the user, the motion of the hand of the user being determined basedon the infrared ray detected by the infrared sensor.
 17. A method ofcontrolling a head-mounted display apparatus worn on a head of a user,the head-mounted display apparatus having a display that displaysinformation, the method comprising: detecting, by an attitude sensor, anattitude of the head-mounted display apparatus; detecting, by amanipulation sensor, a predetermined manipulation performed on thehead-mounted display apparatus; and causing a processor to switch firstdisplay information displayed on the display to second displayinformation according to a change in the attitude detected in thedetecting by the attitude sensor; wherein when the predeterminedmanipulation is detected in the detecting by the manipulation sensor,the first display information being displayed on the display ismaintained without being switched to the second display information evenif there is a change in the attitude detected in the detecting by theattitude sensor.